-
Notifications
You must be signed in to change notification settings - Fork 147
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: ADR Multi-Node Kubernetes Clusters on init #2243
chore: ADR Multi-Node Kubernetes Clusters on init #2243
Conversation
✅ Deploy Preview for zarf-docs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @dgershman,
Thanks for working on this! I read through the k3s documentation to better understand the HA embedded etcd deployment model and have left some comments on the ADR based on that.
Could you clarify the workflow for provisioning the servers? Would Zarf assume the infrastructure is already in place, or would it handle provisioning as well? I think it would be helpful to include a section in the ADR that outlines the end-to-end workflow for users, covering infrastructure setup and deployment.
This results in a line as such for example: | ||
|
||
```ini | ||
ExecStart=/usr/sbin/k3s server --write-kubeconfig-mode=700 --write-kubeconfig /root/.kube/config --disable=traefik --disable=metrics-server --disable=servicelb --tls-san=1.2.3.4 --token=foo |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe we will need to use the cluster-init
flag to enable clustering in HA embedded etcd mode:
To get started, first launch a server node with the cluster-init flag to enable clustering and a token that will be used as a shared secret to join additional servers to the cluster.
https://docs.k3s.io/datastore/ha-embedded
https://docs.k3s.io/cli/server#cluster-options
This is what tells k3s to use the embedded etcd datastore rather than the default SQLite datastore
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes you are right, I wrote this before testing out multiple master
nodes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on the k3s documentation, I believe the minimum steps to create a highly available control plane would be as follows:
-
Launch a server node with the
--cluster-init
flag to enable clustering and a token that will be used as a shared secret to join additional servers to the cluster. -
After launching the first server, join the second and third servers to the cluster using the shared secret and pointing to the first launched server node with the
--server
flag.- The docs state:
An HA K3s cluster with embedded etcd is composed of:
Three or more server nodes that will serve the Kubernetes API and run other control plane services, as well as host the embedded etcd datastore. -
Optionally, add agent nodes to the cluster to run workloads on.
Could you update the ADR to reflect this?
Co-authored-by: Lucas Rodriguez <lucas.rodriguez9616@gmail.com>
based off some v1.0 discussion today, closing this in favor of maintaining a community init package and adding a docs page and linking back to this blog article and repo: |
Description
ADR for
zarf init
multi-node.Related Issue
N/A
Type of change
Checklist before merging